Automated Violin Fingering Transcription Through Analysis of an Audio Recording

We present a method to recuperate fingerings for a given piece of violin music in order to recreate the timbre of a given audio recording of the piece. This is achieved by first analyzing an audio signal to determine the most likely sequence of two-dimensional fingerboard locations (string number and location along the string), which recovers elements of violin fingering relevant to timbre. This sequence is then used as a constraint for finding an ergonomic sequence of finger placements that satisfies both the sequence of notated pitch and the given fingerboard-location sequence. Fingerboard-location-sequence estimation is based on estimation of a hidden Markov model, each state of which represents a particular fingerboard location and emits a Gaussian mixture model of the relative strengths of harmonics. The relative strengths of harmonics are estimated from a polyphonic mixture using score-informed source segregation, and compensates for discrepancies between observed data and training data through mean normalization. Fingering estimation is based on the modeling of a cost function for a sequence of finger placements. We tailor our model to incorporate the playing practices of the violin. We evaluate the performance of the fingerboard-location estimator with a polyphonic mixture, and with recordings of a violin whose timbral characteristics differ significantly from that of the training data. We subjectively evaluate the fingering estimator and validate the effectiveness of tailoring the fingering model towards the violin. In musical instrument performance, deciding the sequence of finger placements needed to produce a given sequence of pitches, known as the fingering, is an important and sometimes difficult problem that musicians need to solve. Fingering decisions can be difficult because the fingering must be both musical and ergonomic, two often-conflicting ideals. For example, the “easiest” fingering on a violin often involves unmusically abrupt changes of timbre. This is because most pitches can be found in more than one location on the instrument (i.e., on more than one string). Each string, however, has a different timbre (for reasons we explain later), and often the easiest fingering involves changes between strings. An experienced musician would choose a balanced fingering that not only satisfies ergonomic finger placements but also expresses the musician’s artistic values. The essence of violin fingering resides in finding both an ergonomic sequence of finger placements Computer Music Journal, 36:3, pp. 57–72, Fall 2012 c © 2012 Massachusetts Institute of Technology. and a musically appropriate fingerboard location sequence, the sequence of locations on the fingerboard on which the finger presses the string. Each fingerboard location specifies both the longitudinal location along the string and the latitudinal position, i.e., which of the four strings is played. Each string is tuned differently and has a distinct timbre. Therefore, it is essential for a violinist to choose a fingerboard-location sequence that sounds musically well-motivated. For example, in Air on the G String, August Wilhelmj’s well-known arrangement of the second movement of J. S. Bach’s Orchestral Suite No. 3, the arranger specifies the entire solo violin part to be played on one string (the G string), most likely to maintain the consistent, warm timbre that is characteristic of the G string. This study aims to develop a method for analyzing an audio recording of a violin and estimating the fingering required to recreate the “sound” of a particular artist’s performance. Such a method would allow a beginner, for example, to analyze the recordings of past masters and gain insights on how to imitate them. It could also help violin students